Search for: All records

We study interactive learning of LLM-based language agents based on user edits made to the agent's output. In a typical setting such as writing assistants, the user interacts with a language agent to generate a response given a context, and may optionally edit the agent response to personalize it based on their latent preference, in addition to improving the correctness. The edit feedback is naturally generated, making it a suitable candidate for improving the agent's alignment with the user's preference, and for reducing the cost of user edits over time. We propose a learning framework, PRELUDE that infers a description of the user's latent preference based on historic edit data. The inferred user preference descriptions are used to define prompts for generating responses in the future. This avoids fine-tuning the agent, which is costly, challenging to scale with the number of users, and may even degrade its performance on other tasks. Furthermore, learning descriptive preference improves interpretability, allowing the user to view and modify the learned preference. However, user preference can be complex, subtle, and vary based on context, making it challenging to learn. To address this, we propose a simple yet effective algorithm named CIPHER that leverages the LLM to infer the user preference for a given context based on user edits. In the future, CIPHER retrieves inferred preferences from the k-closest contexts in the history, and forms an aggregate preference for response generation. We introduce two interactive environments -- summarization and email writing, and use a GPT-4 simulated user for evaluation. On both tasks, CIPHER outperforms several baselines by achieving the lowest edit distance cost while only having a small overhead in LLM query cost. Our analysis reports that user preferences learned by CIPHER show significant similarity to the ground truth latent preferences. 

Declines in the abundance and diversity of insects pose a substantial threat to terrestrial ecosystems worldwide. Yet, identifying the causes of these declines has proved difficult, even for well-studied species like monarch butterflies, whose eastern North American population has decreased markedly over the last three decades. Three hypotheses have been proposed to explain the changes observed in the eastern monarch population: loss of milkweed host plants from increased herbicide use, mortality during autumn migration and/or early-winter resettlement and changes in breeding-season climate. Here, we use a hierarchical modelling approach, combining data from >18,000 systematic surveys to evaluate support for each of these hypotheses over a 25-yr period. Between 2004 and 2018, breeding-season weather was nearly seven times more important than other factors in explaining variation in summer population size, which was positively associated with the size of the subsequent overwintering population. Although data limitations prevent definitive evaluation of the factors governing population size between 1994 and 2003 (the period of the steepest monarch decline coinciding with a widespread increase in herbicide use), breeding-season weather was similarly identified as an important driver of monarch population size. If observed changes in spring and summer climate continue, portions of the current breeding range may become inhospitable for monarchs. Our results highlight the increasingly important contribution of a changing climate to insect declines. 

Monarch butterflies in eastern North America have declined by 84% on Mexican wintering grounds since the observed peak in 1996. However, coarse-scale population indices from northern US breeding grounds do not show a consistent downward trend. This discrepancy has led to speculation that autumn migration may be a critical limiting period. We address this hypothesis by examining the role of multiscale processes impacting monarchs during autumn, assessed using arrival abundances at all known winter colony sites over a 12-y period (2004–2015). We quantified effects of continental-scale (climate, landscape greenness, and disease) and local-scale (colony habitat quality) drivers of spatiotemporal trends in winter colony sizes. We also included effects of peak summer and migratory population indices. Our results demonstrate that higher summer abundance on northern breeding grounds led to larger winter colonies as did greener autumns, a proxy for increased nectar availability in southern US floral corridors. Colony sizes were also positively correlated with the amount of local dense forest cover and whether they were located within the Monarch Butterfly Biosphere Reserve, but were not influenced by disease rates. Although we demonstrate a demographic link between summer and fine-scale winter population sizes, we also reveal that conditions experienced during, and at the culmination of, autumn migration impact annual dynamics. Monarchs face a growing threat if floral resources and winter habitat availability diminish under climate change. Our study tackles a long-standing gap in the monarch’s annual cycle and highlights the importance of evaluating migratory conditions to understand mechanisms governing long-term population trends.